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TuroTwnnS AND APPARATUS FOR 
PERFORMING EFFICIEN r in v ^ 
TRANSFORM OPERATIONS 

Background of the Invention 

• t on relates to inverse transform operations. More 
The present mvent, » rf* * ^ ^ operations 

specifically, the present mvenUon .elates P for 
m0 re efficiently. Still more specially, ^ ^ r block of ^form 

perform^ two-dimensional mverse — oper ■ ^ 

coefficients by using one-dimensional mverse transform 

zero patterns in a block of transform coeffiaents. 

r:t:~r:u > - — — — .rr, 

mak es i. practical to «*nsmi< *• «*— oompression . Generally, 

^pressing data o, further — 7;:7 I ^ on video compression 

gating da.. — ;^: 2 , H,S1, H, 6 Z, H,63, 

schemes. The standards tnclude MPEG 1, ^ ^ key 

„, 63+ , e,c. The standardized — — ^ (for ^ DCT 

algorithm sehen.es: motion eo^ of ^ ^ coefflciems, 
transforms or wavelet/sub-band transforms), q 
5 and variable length coding (VLC). 

, „ encoding removes the temporally redundant 
The motion compensated enc^ ^ ^ 

information inherent ,n ^ ^ sipia ls. Quantization of the 

spatial frequency representation of spa ^ g ^ 

digitized video sample and reduces bt, usage ^ 

oth er factor contributing to reseating ts rf w , s used t0 

ftequently used symbols using code words. In genera, 
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represent a given image determines the quality of the decoded picture. The more bits 
used to represent a given image, the better the image quality. The system that ,s used 
to compress digitized video sequence using the above described schemes is called an 

encoder or encoding system. 

More specifically, motion compensation performs differential encoding of 
frames. Certain frames, such as I-frames in MPEG-2, continue to store the entire 
im age and are independent of other frames. Differential frames, such as B-frames or 
P-frames in MPEG-2, store motion vectors associated with the difference and 
coordinates of particular objects in the frames. The pixel-wise difference between 
objects is called the error term. In MPEG-2, P-frames reference a single frame while 
B-frames reference two different frames. Although this allows fairly high reduction 
ratios, motion compensation is limited when significant changes occur between frames. 
When significant changes occur between frames in a video sequence, a large number of 
5 frames are encoded as reference frames. That is, entire images and not just motion 
vectors are maintained in a large number of frames. This precludes high reduction 
ratios. Furthermore, motion compensation can be computationally expensive. 

Each frame can be converted to luminance and chrominance components. As 
0 will be appreciated by one of skill in the art, the human eye is more sensitive to the 
luminance than to the chrominance of an image. In MPEG-2, luminance and 
chrominance frames are divided into 8x8 pixel blocks. The 8x8 pixel blocks are 
transformed using a discrete cosine transform (DCT) and scanned to create a DCT 
coefficient vector. Quantization involves dividing the DCT coefficients by a scaling 
25 factor The divided coefficients can be rounded to the nearest integer. After 
quantization, some of the quantized elements become zero. The many levels 
represented by the transform coefficients are reduced to a smaller number of levels 
after quantization. With fewer levels represented, more sequences of numbers are 
similar For example, the sequence 4.9 4.1 2.2 1.9 after division by two and rounding 
30 becomes 2 2 1 1. As will be described below, a sequence with more similar numbers 
can more easily be encoded using either arithmetic or Huffman coding. However, 
quantization is an irreversible process and hence introduces significant loss of 
information associated with the original frame or image. 
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Huffman or arithmetic coding takes the most common .o„g sequences of 
numbers of bits and repiaces them wirh a shorter sequence of numbers or btt, Agatn, 
Huffman or arithmetic coding is limited by common sequences of numbers or bn, 
Sequences that contain many different numbers are more difficult to encode. 

Currently available compression techniques for compressing video or image 
data use transform and inverse transform operation, However, transform and tnverse 
transform operations are computationally expensive and introduce delay tnto Ume 
sensitive data stream, The transform and inverse transform operations are often used 
in transcoding systems for scaling a data stream associated with one se, of ban wtd h 
requirements to a modified data stream associated with another set of bandwtdth 
requirement, Transform encoded data is often rescaled to meet bandwidth linutattons. 
Transform and inverse transform operations are often a bottleneck transcoding system, 
,t is therefore desirable to provide techniques for efficiently performing mverse 
transform operation, Techniques for efficiently performing inverse transform 
operations could be particularly useful in transcoding systems. 
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Summary of the Invention 

According to the present invention, methods and apparatus for p— « 
effl cien, inverse transform operations are provided. Zero patterns are *. . « - 
blo ck of transform coefficients to derive zero pattern mformatton. Ustng the zero 
^1 m—n, one-d— inverse transform operations 
Lset of the coiumns of the block of transform coeffictents. Zero patterns ca 
! o U * identified in the resulting hlck. One-dimenstona, mverse transform 
l - then performed on the rows of the resulting biock to compiete the *vo- 

operations can be performed firs, on the rows and then on the cohunns. 

h one eiamp.e, the biock of transform coefficients is an MPEG 
One-dimensiona, tnverse transforms can be performed first on a subse o f the - 

of — » .be ;r of ^j^zzzzr* 

performed first on the columns or on the rows. 

Another aspect of the invention provides an apparatus for performing one- 
dimenstW tnvJe transforms on the rows and columns of a 
coefficients. The apparatus includes memory and a processor coupled w tt> « 
The processor is configured to identify zero pattern informal assorted wrth *e 
Zl of transform coefficients, perform one-dimensional inverse transforms 
H of rows and columns of the block of transform coefficients usm g the zero 
pattern information. 

Another aspect of the invention pertains to computer program products 
„ incudint machine readable medium on which is stored program instructs, tab,, 
m data structures for implementing a method as described above. Any o, 

instructions mat can be provided on such computer readable merha. 
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„ A' „ of the nature and advantages of the present invention 
A further understanding of the nature speci f lca tion and the 

* n tv.f remaining portions ot the speuu^ 
may be realized by reference to the remaining p 

drawings. 
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Brief Description of the Drawings 

Figur e I is di^-n-c representation of a system .ha, can use the — s 

u ■ the nresent invention, according to specific embodiments. 

transform operations on rows and columns of a block. 

Figure 6 is a diagrammatic representation of a system that can 
implement the present invention. 
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Detailed Description of Specific Embodiments 

The present invention gene* relates to data concession. 

• „ bribed eenerally in m^mCom^LBook, by Mark Nelson 

techniques are described generally in — u 

(ISBN: 1558514341). the entirety of which is hereby incorporated by reference 



purposes. 



Many technic^ for data compression - — * — ' ^ ~ 
for video compression is MPEG-2. MPEG-2 uses motion compensate, a* 
le — , nation, an. — — * - ^ — * 

moI e computational expensive aspects of data compress sch -» « Mh 
Lsform Rations that are used to convert transform coeffcents mto da. ue. 
Worm operations are used durin g sever, stages inc,udin g ™ — ^ 
, H H,ta Rescaling data is described in Paten. Application No. 09/766,020 titled 
encoded data. Reseating r „ moresse d Video Data at the date of 

Methods For Efficient Bandwtdth Seating Of Compressed V 

fi Ung on January .8, 200!, the entirety of which is hereby tncorporated by reference 

for all purposes. 

* present invention provides technics for improving upon 

• ♦ fnrm operations The techniques of the present 

rrr— " — - - - — - * -» °rt: 

Zlns in one examp.e, the present invention breaks down a two-dtmenstonal 
rerisform operation into one-dimensiona, inverse transform operations. On. 

Is of a b.oc* of transform coefficients. By appiying inverse transform operation 
Z on seiected coiumns and rows, the technic of the p^ent — can 

Pigure , is a diagrammatic representation of a system having inverse transform 
operatil that can use the technics of the present invention, aceordmg to vanou 
IdLus. .though Figure , w,H be described in detatl to show a system for 
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rescaling data, the — of the present — are completely genera,, an, can 

requan.iza.ion vta one m °" ^ by receiving compressed video data 

at 121. lhe vmeu u motion compensation 128. 

• 194. inverse transform coding 126 and mono 

dequantization 124, inverse ur«u p.frames and B-frames 

Motion compensation 12S htc.udes an — <^ ^ processing me 

are reconstruct using a Restore memory ^ encoding l36 . Por 

vW eo data wim^sfor. coding 1 32,re q ua„.-o„ 1 ^ 

bit rate reduction of .he video da* requantrzatron 134 - perform 

quantization step value. 

,• l n and requantization 134, each ima g e is decoded by 
After transform codtng 132 and requa compensa .ion 142 

dequantization 13S and inverse reform codtng .«> W« m ^ 

. filestore memory 144. Motton P ^ 
„ is summed 146 with the nex, decoded ptchrre 148 f motion 

132 , requantization 134, and ^ „ ^ m m d .36 

compensation 142 with moUon vector rKeived at 12 1. 

produces compressed video data 1 50 havmg a lower b„ rate 

A tressed video data 150 may then be combined with other 
" t and — ed onto a channel. Br* - - 

requanfzed compressed vrd ^ ^ ^ 

reference I and Pictures is represen ed * * £f 

memories 130 and 144 due to requa„t,za„on. The mo P ^ ^ 

scbeme ,20 ta.es into ^^^^ ra ,e reduction ratio is 
30 provides quality video re-encodmg, operatio ns are required a, 126 
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.pensive. « is desirable ,o provide the efficient — es of the present invention 
for performing inverse transform coding operations. 

Fial re 2 is a diagrammatic representation of another example of a system that 
Figu a 8 . ,. „ Fimre 2 shows decoding using inverse 

— :m d" - video data is men decode, coning 

motion compensation 268. iviouon p 

u fi- <; and B frames are reconstructed using a framestore memory 270. The 
where P-frames and B-framesar The decoded fram es can be provided 

decoded pictures or frames are provided at 188. lhe a 
to an output device for display. 

Ptgnre 3 is a diagrammatic represent showing the application of an inverse 

transform on a block of transform coefficients. A transform coefficen t Mo* 301 
transiorm uu« ^\ 1 and 313. 

eontains rows 325, 327, 329, 331, and 333 and columns 303 30 307 ^ 
though me bloc, of transform coefficients is shown as a five by 
be understood that transform coefficients blocks can have a range of different -e. 
; !lTin the bl^s are a,so provided for illustrative purposes and do no, 

Tgllycorrespondtomehighfre.uencycomponentsoft. 

Toapplyaninversetransfonnonthefransformcoeffrcientbloc^ 
30 To apply an & ^ o _ dimenslonal 

2 , a two-dimensional inverse DCT is typically PP 

nrT however does not take advantage of the DCT coettic 
mverse DCT, however, u calculates the 

■ • • uwir Annlvine a two-dimensional inverse ^ 
characteristics in block 301. Appiyi"& 
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for calculating the inverse DCT. 



where 

C (x) = i if x is °' else 1 if x > 0; md 

a/2 

N is the number of rows or columns in a matrix. 



\2x + \)irt 



IN 



cos 



\2y + \)j 



IN 



(Equation 1) 



Ahnough .here may be speciahzed hardware for casing two-dimensional 

expensive. For time sensitive content such as rea.-t.me vtdeo, any de.ay may 
undesirable as it may be detectable by a viewer. 

nation 2 shows the breakdown of .he two-dimensiona. inverse DCT into 
separa .e one-dimensiona, inverse DCT opera.io„s applied to .he rows and — of 
the block of transform coefficients 301. 



| AM 

Pixel(x,y)=- 7 =S C(i) 



where 



^CU)DCT(iJ)cos 



2N 



cos 



(2jc + 1)/^ 



IN 



20 



C (x) = 4= if x is ° 5 else 1 if x > 0; and 

a/2 

N is the number of rows or columns in a matrix. 



(Equation 2) 



25 



Equation 2 can be used .o ca.cu,a.e me one-dimensiona. inverse ttansform on 
Equation ^ one -dimensional inverse ttansform 

,he coefficients offlie columns of block 301. Theonea „„ therowso f 
on the co.umns can be followed by one-dimensional inverse ttansform or * - 

Z v** or vice versa One-dimensional inverse DCT operations applied to the rows 
block 30., or vice ve^ ^ ^ ^ ^ g wo . 

and co.umns of a ^ . ^ of , ransform ^ients. Like 

dimensional inverse DCT operations appn ooer ations 
two-dimensiona. inverse DCT operations, one-din— inverse DCT opera,, 
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can also be computationally expensive. I, would be useml .o ™ *^ ° 
one-dimensiona, inverse DCT operations performed. One way of reductng the num* 
of one-dimensional inverse DCT operations is to analyze the characters o h 
block of — coefficient, Performing one.—! inverse DCT op^Uons 
on . row or coiumn of zeros yields a row or coiutnn of zeros. In add,tio„, the one- 
dimensiona, inverse DCT operations on a row or co.umn of near zeros can be 
approximated by a row or column of zeros. 

Consequently, one-dimensional inverse DCT operations do not need to be 
performed on rows or columns containing all zeros values or values close to zero 
Values close to zero or near zero can be determined by identifying values below a se 
factor such as 2. Alternatively, values near zero may be determine as less than »e 
percent of the average or maximum of all values. The number of one-dmtens^ 
nverse DCT operations performed on a block of transform coefficients can be reduced 
by identifying zero patterns in the block. By identifying zero patterns and denvrn 
L pattern information, one-dimensiona, inverse DCT operations can be perform d 
on a subset of the rows and columns of ablock of DCT transform coefficients wh„e 
preserving the information contained in the DCT transform coefficient block. 

According to various embodiments, the techniques of the present invention 
contemplate identifying a zero pattern in block 301 to acquire zero pattern information 
The zeros inb,oek301 are located primarily colum„s311 and3,3. A row of zeros can 
a,so be located a, 333. As will be appreciated by one of skill in tire art, Mocks 
coefficients may con* a number of zeros in the lower right hand of ^b'ock 

, when only low frequency information is repr.en.ed in the portion of the fram, A 

coefficient block containing only a few DCT coefficients in the upper left hand comer 
Zero patterns are also predominant during realization or even dunng *** 
quantization with a high quantization scale. Zero patterns are a,so frequent when fl» 
,„ are applied to blocks of transform coefficients. As will be appreciated by one of sk,U 
I ,h ear, zero patterns exist in a variety of different b.ocks including b ocks from 
MPEG .-frames, P-^es, and B-frames. Applying filters to b,ocks of 
coefficients is described in U.S. Patent Application No. _/__ (Attorney Docket 
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No CISCP219) by Wu et al., and titled Methods and Apparatus for Transform 
Coefficient Filtering, the entirety of which is incorporated by reference for all 



purposes. 
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Generally, determining the location of zero values or near zero values in a block 
of transform coefficients is referred to herein as identifying zero patterns and zero 
pattern information. In block 301, zeros are found primarily in columns 311 and 313. 
One-dimensional inverse transform operations are performed on column 303, 305, and 
307 to yield the values in columns 315, 317, and 319 respectively. Columns 321 and 
323 in block 347 are populated with zeros. To calculate the actual pixel values 
associated with block 347, the one-dimensional inverse DCT transform operations are 
applied to all of the rows of block 347. As shown, the one-dimensional inverse DCT 
transform operations are applied to row 335, 337, 339, 341, and 343 at 349 to yield the 
actual pixel value block 351. Consequently, one-dimensional inverse DCT transform 
operations are performed on a subset of the rows and columns of the block of transform 
coefficients. Instead of performing inverse transform operations on all columns 303- 
313 and all rows 325-333, one-dimensional inverse DCT transform operations are only 
performed on columns 303-307 and columns 325-333. 

Figure 4 is a diagrammatic representation showing application of one- 
dimensional inverse transform operations on selected rows of block 401. Identifying 
zero pattern information in block 401 indicates that zeros can be found primarily m two 
rows 431 and 433 and in one column 413. There are more rows of zeros than there are 
columns of zeros. Generally, rows of zeros or columns of zeros refer to rows or 
columns containing mostly zeros or values close to zero. Fewer one-dimensxonal 
inverse DCT operations will be performed if the one-dimensional inverse transforms 
are calculated for the rows first instead of the columns first as shown in Figure 3. 
Using the zero pattern information, one-dimensional inverse transforms are performed 
on selected rows 425, 427, and 429 that represent a subset of all rows in the block 401 
to yield rows 435, 437, and 439 of block 447. Rows 441 and 443 are populated w,th 
zeros. According to various embodiments, zero pattern information can be obtained for 
block 447. Column 417 contains values that are close to zero. As noted above, a one- 
dimensional inverse transform applied to a column of values close to zero can be 
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approximated by a column of zeros. If, as one examp.e, a factor » ^ - « 
.Lhold .o define near-zero columns, column 4,7 can be filtered as - 
column. One-dimensiona, inverse U—s can «he„ be apphed ,o co.umns 415. 4.9, 
421 , and 423 to yield pixel block 45 1 . 

Figure 5 is a process flow diagram showing the caption of one-d,mension* 
averse transforms on selected rows and columns of a block of transform coeffictents. 
5 will be described with reference to Figure 4. A, 501 the next transform 

40! The zeros are found primarily in rows 43! and 433 and » column 413. A, 505 , 
lermined whether mere are more columns or rows of zeros. Accords o vano, 
embodiments, zeros include values of near zero such as 3 or 5. Smce block 40 has 
r!ls of eros and on,y one column of zeros, one-dimensiona, inverse transfonn 
1 performed on selected rows 425, 427, and 429. At 5,5, zero pattern mformatton 
I! Ik 447 can optional be tden.if.ed. .dentins zero pattern informal for 
block 447 indicates that zeros are located in column 417. 

One-dimensiona, inverse transforms are performed on columns 415, 4,9, 421 

, • , ui The next block of transform coefficients is 

and 423 to yield actual pixel block 451. 1 he next diock 

ZZZ a. 50, « should be noted tna, the process flow described m tgnre 5 
Is not need to be performed in any particular science. For examp.e, one- 
dlnenla, inverse transforms may be performed on a,l of the selected rows of a 
"Lsform coefficients sin—ly. The iden.inca.ion of zero pattern 
Nation a, 509 and a, 5,5 are optional. A determinanon a, 505 ^mer there » 
le columns or rows of zeros ts also optional, and a system using , he e— o 
the present invention can be configured to perform one-dimens.ona, — - 
co,ls or rows first regard.ess of zero partem information. !n one example * 
Tystem can he configured to perform one— al inverse transform op^tonson 
a subset of .he total number of columns using zero pattern tnformafco. One 
diliona, inverse transform operates are men performed on a„ the rows wtthout 
regard to zero pattern information. 
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The present invention for filtering transform coefficients to provide rate 
reduction in a bitstream can be implemented in various network systems. In vanous 
embodiments, this can be implemented in the headend of high bandwidth networks 
such as a cable network or a satellite network, In the context of a cable network, the 
invention ,s implemented to a s,anda,one system, such as Cisco 6920 RateMux® 
available from Cisco Systems, Inc. or in a line card of a cable modem headend such as 
the Cisco UBR 7200 also available from Cisco Systems, Inc. 

Figure 6 depicts the basic components of a cable modem headend that can be 
used to implement the present invention, according to specific embodiments. Although 
the techniques of the present invention can be integrated into a cable modem headend, 
the present invention can also be used in a standalone system. Figure 6 shows an 
implementation using the cable modem headend. 

A Data Network Interface 602 is an interface component between an external 
data source and the cable system. External data sources transmit data to data network 
interface 602 via optical fiber, microwave link, satellite link, or through various other 
media Also as mentioned above, a Media Access Control Block (MAC Block) 604 
receives data packets from a Data Network Interface 602 and encapsulates mem wtth a 
MAC header. 

m a specific embodiment as shown in Figure 6, CMTS provides functions on 
three network layers including a physical layer 632, a Media Access Control (MAC) 
,ayer 630, and a network layer 634. Generally, the physical layer is responstble for 
' receiving and transmitting RF signals on the cable plan, Hardware portions of the 
physical layer include a downstream modulator and transmitter 606 and an upstream 
demodulator and receiver 614. The physical layer also includes software 686 for 
driving the hardware components of the physical layer. 

Once an information packet is demodulated by the demodulator/receiver 614, it 
is then passed ,0 MAC !ayer 630. A primary purpose of MAC layer 630 ts to 
encapsulate and decapsulate packets within a MAC header, preferably accordmg to the 
above-mentioned DOCSIS standard for transmission of data or other information. 
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MAC layer 630 includes a MAC hardware portion 604 and a MAC software 
portion 684, which function together to encapsulate information packets with the 
appropriate MAC address of the cable modem(s) on the system. After the upstream 
information has been processed by MAC layer 630, it is then passed to network layer 
634 Network layer 634 includes switching software 682 for causing the upstream 
information packet to be switched to an appropriate data network interface on data 
network interface 602. 

When a packet is received at the data network interface 602 from an external 
source, the switching software within network layer 634 passes the packet to MAC 
layer 630 MAC block 604 transmits information via a one-way communication 
medium to downstream modulator and transmitter 606. Downstream modulator and 
transmitter 606 takes the data (or other information) in a packet structure and converts 
it to modulated downstream frames, such as MPEG or ATM frames, on the 
downstream carrier using, for example, QAM modulation (other methods of 
modulation can be used such as CDMA (Code Division Multiple Access) OFDM 
(Orthogonal Frequency Division Multiplexing), FSK (FREQ Shift Keying)). The 
return data is likewise modulated using, for example, QAM 16 or QSPK. Data from 
other services (e.g. television) is added at a combiner 607. Converter 608 converts the 
modulated RF electrical signals to optical signals that can be received and transmitted 
by a Fiber Node 610 to the cable modem hub. 

It is to be noted that alternate embodiments of the CMTS (not shown) may not 
include network layer 634. In such embodiments, a CMTS device may include only a 
physical layer and a MAC layer, which are responsible for modifying a packet 
according to the appropriate standard for transmission of information over a cable 
modem network. The network layer 634 of these alternate embodiments of CMTS 
devices may be included, for example, as part of a conventional router for a packet- 
switched network. 

In a specific embodiment, the network layer of the CMTS is configured as a 
cable line card coupled to a standard router that includes the physical layer 632 and 
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MAC layer 630. The techniques of the present invention including a filter stage and 
rate control stage shown in Figure 1 can be implemented on a Hne card. Using Uns 
type of configuration, the CMTS is able to send and/or receive IP packets to and from 
the data network interface 602 using switching software block 682. The data netwo* 
interface 602 is an interface component between external data sources and the cable 
system The external data sources transmit data to the data network interface 602 vta, 
for example, optica! fiber, microwave link, satellite link, or through various med„ 
The data neUvork interface inc.udes hardware and software for interfacing to vanous 
networks such as, for example, Ethernet, ATM, frame relay, etc. 

As shown in Figure 6, the CMTS includes a hardware block 650 including one 
or more processors 655 and memory 657. These hardware components interact wrth 
software and other hardware portions of the vanous layers within the CMTS. Memory 
657 may include, for example, I/O memory (e.g. buffers), program memory, share 
memory, etc. Hardware block 650 may physically reside with the other CMTS 
components. 

m one embodiment, the software entities 682, 684, and 686 are implemented as 
part of a network operating system running on hardware 650. Further, the provisions 
of this invention for providing quality of service for multicast streams are preferably 
implemented in software as part of the operating system. 

Because such information and program instructions may be employed to 
implement the systems/methods described herein, the present invention relates to 
machine readable media that include program instructions, state information, etc. or 
performing various operations described herein. Examples of machine-readable medra 
include, but are no, limited to, magnetic media such as hard disks, floppy dtsks, and 
magnetic tape; optical media such as CD-ROM disks; magneto-optica, medra such as 
optical disks; and hardware devices that are specially configured ,0 store and perform 
, program instructions, such as read-only memory devices (ROM) and random access 
memory (RAM). The invention may also be embodied in a carrier wave traveling 
over an appropriate medium such as airwaves, optical lines, electric hues, etc. 
Examples of program insurious include bom machine code, such as produced by a 
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compiler, and files containing higher level code that may be executed by the computer 
using an interpreter. 

While the invention has been particularly shown and described with reference 
to specific embodiments thereof, it will be understood by those skilled in the art that 
changes in the form and details of the disclosed embodiments may be made without 
departing from the spirit or scope of the invention. For example, the embodiments 
described above may be implemented using firmware, software, or hardware. 
Moreover, embodiments of the present invention may be employed with a variety of 
communication protocols and should not be restricted to the ones mentioned above. 
For example, the headend has a variety of embodiments which include a cable modem 
termination system coupled to a router or a multicast router. A cable modem can also 
be a separate entity or entirely integrated into a client system. In addition and as 
mentioned above, the invention may be implemented in both differential and single- 
ended configurations. Therefore, the scope of the invention should be determined with 
reference to the appended claims. 
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